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"Knowing  where  you've  come  from  and  where  you  are  is  essential  to  knowing  how  to 
get  where  you  want  to  go.  Developing  a  new  generation  of  products  is  a  lot  like  taking  a 
journey  into  the  wilderness.  Who  would  dream  of  setting  off  without  a  map?"  1 

Steven  C.  Wheelwright  and  W.  Earl  Sasser,  Jr. 


INTRODUCTION 

This  paper  proposes  a  product  development  methodology  (PDM)  for  complex  systems 
evolving  within  the  current  economic  climate  of  the  United  States,  as  well  as  die  unstable  state  of 
world  affairs  envisioned  throughout  the  decade.  The  PDM  facilitates  the  development  of  systems 

that  are  "multipurpose,  flexible,  highly  mobile,  and  incorporate  maximum  bang  for  the  buck."2 

Ironically,  the  unstable  nature  of  the  development  environment  within  the  defense  community 
parallels  the  one  encountered  by  the  commercial  sector  over  the  last  20  years.  Successful 
companies  have  responded  by  adopting  a  product  development  methodology  that  adapts  to  ever 
changing  market  demands  and  the  concern  for  near  term  returns  (profit). 

The  PDM  exploits  lessons  learned  from  the  commercial  market  analogy  to  establish  a  flexible, 
low  risk,  cost  effective  approach  for  technological  progress.  The  approach  suits  systems 
development,  especially  those  involving  complex  mission  critical  computer  systems. 

Examination  of  the  commercial  product  development  process  reveals  a  strategy  that  can 
achieve  the  procurement  flexibility  needed  by  DoD.  This  strategy  concentrates  on  leveraging  the 
state-of-the-art  in  a  cost  effective  manner.  The  strategy  also  addresses  risk  management. 

The  key  to  the  technological  success  of  this  strategy  relies  on  an  incremental  development 
process.  The  IBM  PC  serves  as  a  perfect  example.  The  i486  based  PC  resulted  from  successful 
sales  of  the  286, 386SX  and  i386  based  versions.  Incremental  upgrades  enabled  IBM  to  respond 
to  changes  in  market  demand  as  well  as  facilitate  the  transition  of  the  state-of-the-art.  In  this 
manner,  IBM  attained  strategic  flexibility. 

The  discussion  starts  at  a  basic  level  and  progresses  to  a  macroscopic  perspective.  This  paper 
contains  four  parts:  adaptation  of  a  commercial  approach,  incorporation  into  an  overall  risk 
management  scheme,  application  to  an  open  architecture  transition,  and  a  summary. 

The  paper  recommends  using  open  architectures  and  commercial  off-the-shelf  (COTS)  items 
to  implement  the  incremental  improvements  outlined  by  the  PDM.  The  summary  includes 
guidelines  for  successful  product  development,  as  well  as  ideas  for  future  work  in  this  area. 


ADAPTATION  OF  A  COMMERCIAL  APPROACH 

Decreasing  commercial  product  life  cycles  have  required  technologies  to  be  developed  at 

faster  rates.3  As  a  result,  companies  have  devoted  more  effort  to  the  product  development 
process.  The  process  differs  from  company  to  company;  however,  the  high  tech  arena  focuses 
on  the  time  to  market.  Shortening  the  time  to  market  enables  a  company  to  increase  market  share, 
adapt  product  characteristics  to  market  needs,  enjoy  high  margins  typically  encountered  in  the 

beginning  of  a  product’s  life  cycle,  and  shorten  the  payback  period.4 
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This  focal  point  requires  a  company  to  make  decisions  in  what  Preston  Smith  refers  to  as  the 

"fuzzy  front  end"  of  the  product  development  process.5  Lack  of  quantitative  information  and 
organizational  structure  characterize  the  ambiguity  of  the  "fuzzy  front  end."  Decisions  made  at 
this  time  greatly  affect  the  product's  evolution.  The  expensive  nature  of  changes  made  down  the 
road  heightens  the  significance  of  these  up  front  decisions.  As  a  product  progresses  from 

planning,  through  design,  production,  test  and  delivery  the  cost  to  correct  an  error  increases.6 

Smith  offers  a  simple  decision  analysis  technique  to  attack  problems  in  the  "fuzzy  front  end." 
The  technique  concentrates  on  the  interrelationships  between  time,  development  cost, 
performance  and  profit.  He  prefers  an  approach  based  on  estimates  generated  quickly.  Smith 
believes  complicated  estimation  tools  waste  time  and  lead  to  a  false  impression  of  the  accuracy  of 
the  available  data.  His  book  presents  several  examples  to  demonstrate  the  merit  of  his  approach. 
Therefore,  this  paper  concentrates  on  the  adaptation  of  Smith's  ideas  to  complex  systems. 

At  first  glance,  the  lack  of  the  profit  motive  within  the  government  appears  to  create  an 
obstacle  to  the  application  of  Smith's  approach  to  the  development  of  complex  systems. 
However,  consider  the  savings  the  government  can  pursue  when  building  systems.  This 
viewpoint  establishes  the  profit  motive;  when  systems  cost  less,  profits  from  savings  follow.  In 
other  words,  life  cycle  cost  savings  create  a  profit.  The  analogy  between  profit  and  cost  savings 
permits  a  modification  to  Smith's  model  for  product  development  in  the  defense  sector.  Figure  1 
illustrates  the  product  development  framework  that  results  when  life  cycle  cost  replaces  profit  in 
Smith's  model.  The  arrows  indicate  relationships  between  the  areas  identified  in  the  circles. 


Figure  1.  Product  Development  Framework 

Note,  this  model  separates  development  costs  from  life  cycle  costs.  This  definition  differs 
from  the  definition  of  life  cycle  cost  used  in  Naval  acquisitions.  For  Naval  acquisitions,  life  cycle 
cost  is  the  sum  total  of  the  direct,  indirect,  recurring,  non-recurring,  and  other  related  costs 
incurred,  or  estimated  to  be  incurred  in  the  design,  research  and  development  (R&D),  investment, 

operation,  maintenance,  and  support  of  a  product  over  its  life  cycle.7 

The  product  development  framework  fosters  decisions  based  on  time,  performance, 
development  cost  and  life  cycle  cost  tradeoffs.  To  achieve  savings,  the  product  development 
framework  must  assume  a  baseline  for  time,  performance  and  cost.  An  existing  system  functions 
as  the  baseline.  The  baseline  system  establishes  cost  and  performance  ceilings.  Measure  time, 
performance  and  cost  in  terms  of  the  incremental  contribution  to  the  baseline  system  when 
developing  new  products  for  existing  systems. 
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This  paper  uses  qualitative  reasoning  to  demonstrate  the  utility  of  the  product  development 
framework.  A  color  coding  scheme  depicts  the  incremental  contribution  for  each  area.  Ideally, 
the  color  coding  scheme  would  use  a  traffic  light  pattern.  Picture  red  for  an  undesirable  rating, 
yellow  for  an  indeterminate  condition  and  green  for  a  favorable  estimate.  Figure  2  exhibits  the 
alternate  color  coding  scheme  used  in  this  paper  to  facilitate  duplication  of  the  material. 


Decreased  performance, 
increased  cost,  or 
increased  time  (red) . 

Indeterminate  (yellow) . 

O  Increased  performance, 
decreased  cost,  or 
decreased  time  (green) . 


Figure  2.  Color  Coding  Scheme 

A  quantitative  analysis  would  eventually  replace  qualitative  reasoning.  The  progression  of 
time  enables  the  analysis  to  improve  as  more  information  becomes  available  and  decisions  are 
revisited.  Hence,  the  quantitative  analysis  gets  fine  tuned  as  the  product  becomes  well  defined. 

This  product  development  framework  facilitates  the  assessment  of  life  cycle  costs, 
development  costs,  and  development  time  for  specific  performance  requirements.  Continued 
appraisal  will  result  in  a  set  of  performance  requirements  that  meets  cost  goals. 

One  problem  not  represented  directly  in  the  framework  is  the  difficulty  mustering  support  for 
the  acquisition  of  systems  on  the  basis  of  life  cycle  cost.  Opponents  can  attack  the  fidelity  of 
forecasts  beyond  a  5  year  period. 

However,  a  shorter  product  development  cycle  addresses  this  problem  by  trimming  the 
payback  period.  The  payback  period  is  the  time  it  takes  to  recoup  the  initial  development  cost 
through  life  cycle  cost  savings.  Reduced  payback  periods  strengthen  life  cycle  cost  estimates. 
The  incremental  product  development  approach  capitalizes  on  condensed  payback  periods. 

The  framework  addresses  issues  on  a  discrete  product  basis.  Examples  of  discrete  products 
include:  disk  drives,  power  supplies,  and  stand  alone  computers.  In  contrast,  complex  computer 
systems  represent  an  amalgam  of  discrete  computer  products.  They  require  a  technique  that 
weighs  each  decision  on  a  macroscopic  level.  The  four  element  diagram  cannot  guide  complex 
decisions  without  a  higher  level  of  abstraction.  The  next  section  outlines  the  higher  level. 


RISK  MANAGEMENT  FOR  COMPLEX  COMPUTER  SYSTEMS 

Many  discrete  technical  approaches  compete  for  attention  in  the  "fuzzy  front  end"  of  complex 
computer  systems  development.  The  aforementioned  product  development  framework  expedites 
decisions  on  a  case  by  case  basis,  but  cannot  manage  a  complex  computer  system  in  its  entirety. 
A  useful  methodology  must  provide  a  map  for  macroscopic  considerations.  This  consideration 
differentiates  Smith's  commercial  technique  from  complex  systems  development  Nevertheless, 
Smith's  framework  forms  a  foundation  for  the  map  used  in  the  higher  level  of  abstraction. 

Basically,  the  methodology  sets  the  stage  for  each  discrete  approach  to  compete  in  terms  of 
time,  cost  and  performance.  A  three  step  process  establishes  a  clear  path  from  the  "fuzzy  front 
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end"  through  product  definition  to  a  low  risk  development  scenario.  Figure  3  illustrates  this 
process  for  the  development  of  a  complex  computer  system  on  the  basis  of  cost. 


Item  l 


Incorporate  into 
ADM 


Schedule  for 
Technology 
Insertion 


Item  N 


Reconsider 
As  Commercial 
Market  Matures 


INCREASED  PRODUCT  DEFINITION 


*  Note  -  Subjective  model  relative  to  existing  baseline. 


Figure  3.  Product  Development  Methodology 

The  first  step  identifies  each  discrete  candidate.  Identification  requires  at  least  a  subjective 
description  in  terms  of  time,  performance,  life  cycle  cost  and  development  cost.  In  fact, 
subjective  approaches  accelerate  the  process.  A  plethora  of  candidate  approaches  typically 
overwhelms  the  front  end  of  complex  computer  system  development.  A  detailed  quantitative 
analysis  of  each  would  consume  time  and  money. 

A  RAND  study  of  process  plants  demonstrates  the  lack  of  accuracy  of  data  in  the  "fuzzy  front 
end."  Process  plant  estimates  generated  on  the  basis  of  R&D  data  alone,  can  easily  overrun 
budgets  by  100%.  As  the  level  of  project  definition  and  quantity  of  engineering  data  increase, 

overruns  decline  to  about  10%  at  a  full  cost  design  stage.8 

Ensure  early  efforts  focus  on  the  rapid  development  of  high  potential  products,  rather  than  up 
front  detailed  cost  analyses.  Get  products  into  existing  systems  quickly.  Keep  the  up  front 
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analysis  simple  to  reduce  development  costs.  Mitigating  costs  reduces  risk.  In  the  long  term,  the 
incremental  product  development  methodology  promotes  a  diversified  development  portfolio. 

Breaking  down  the  complex  system  development  into  a  step-by-step  sequence  of  limited 
challenges  contains  risk  and  cost.  Northern  Telecom's  venture  from  analog  into  digital  switching 
systems  for  the  telecommunications  industry  serves  as  an  example.  Instead  of  going  for  the  local 
DMS  100  switch  right  away,  the  company  started  with  the  development  of  a  PBX  (private  branch 
exchange),  which  gave  it  a  base  for  understanding  important  new  technologies,  digitization 
techniques,  advanced  programming  languages,  and  network  design.  Treating  the  effort  as  a  step- 
by-step  sequence  of  more  limited  challenges  allowed  Northern  Telecom  to  contain  its 

development  risk  and  keep  development  costs  from  going  through  the  roof.9 

The  second  step  involves  a  discussion  of  each  candidate  approach.  The  discussion  includes 
establishing  fundamental  criteria  for  advanced  development,  technology  insertion  and  future 
consideration.  Discussion  challenges  the  subjective  nature  of  the  data. 

The  third  and  final  step  sorts  the  candidates  into  those  considered  for  immediate  advanced 
development,  future  technology  insertion,  and  further  consideration.  At  this  part  of  the 
development  process  the  high  priority  candidates  require  a  detailed  quantified  analysis. 

Depending  on  cost  goals,  products  can  move  to  and  from  the  advanced  development  model 
(ADM)  and  technology  insertion.  Existing  ADM  products  replaced  by  candidates  from  the 
technology  insertion  area  act  as  contingencies;  they  create  a  backup  in  case  of  product  failures. 

The  identification,  discussion  and  prioritization  (IDP)  of  discrete  candidates  lead  to  system 
definition.  Figure  3  shows  how  development  proceeds  from  the  "fuzzy  front  end"  to  clear-cut 
product  definition.  The  process  mitigates  risk  by  giving  priority  to  approaches  that  yield  the 
biggest  cost  savings  with  the  shortest  payback  period.  As  time  progresses  the  product  becomes 
well  defined  and  information  is  available  to  make  decisions  quantitatively  rather  than  qualitatively. 


APPLICATION  OF  THE  PPM  TO  AN  OPEN  ARCHITECTURE  TRANSITION 

Traditionally,  combat  system  development  requires  a  quantum  leap  in  performance.  Figure  4 
exemplifies  the  computing  throughput  required  by  expanding  sensor  array  configurations. 


Figure  4.  Quantum  Leap  in  Processing  Requirements 
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In  addition  to  greater  computational  requirements,  this  trend  also  creates  demands  for  faster 
communication  links  and  denser  memory  configurations.  One  approach  to  meeting  these 
demands  involves  the  incorporation  of  an  open  architecture.  Open  architectures  leverage  fast 
paced  commercial  technology  development  by  maintaining  compatibility  with  commercial 
standards. 

Case  histories  show  building  off  existing  foundations  of  core  technologies  generates  success 
in  the  commercial  sector.  Companies  that  focus  new  products  on  extensions  to  a  single  key 
technology  are  far  more  successful  than  those  that  pursue  technical  diversity.  "The  test 
opportunities  for  rapid  growth  come  from  building  an  internal  critical  mass  of  engineering  talent 
in  a  focused  technological  area,  yielding  a  distinctive  core  technology  that  might  evolve  over  time, 

to  provide  a  foundation  for  the  company's  product  development."10 

Accordingly,  the  transition  from  an  existing  sensor  processing  system  to  an  open  architecture 
based  system  offers  a  prime  example  of  the  utility  of  the  PDM  proposed  in  this  paper.  The 
combination  of  the  PDM  and  open  architecture  philosophies  facilitates  future  technology  upgrades 
for  the  sensor  system.  Hardware  and  software  commonality  contain  costs. 

Figure  5  shows  an  open  architecture  for  a  sensor  processing  system.  The  key  features  of  this 
architecture  are  the  sensor  distribution  network,  the  data  distribution  network  and  the  common 
processing  cabinets.  The  common  processing  cabinets  fit  into  the  open  architecture  scheme  by 
utilizing  a  commercially  available  bus  architecture  for  the  backplane.  Any  vendor  can  integrate 
equipment  into  the  system  as  long  as  they  adhere  to  the  interface  standards. 
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The  lack  of  mature  standards  poses  an  obstacle  to  the  implementation  of  open  architectures. 
For  example,  many  of  the  Next  Generation  Computing  Resources  (NGCR)  initiative’s  interface 
standards  have  not  been  written.  Therefore,  cost  and  performance  are  indeterminate. 

In  addition,  many  existing  combat  systems  do  not  possess  open  architecture  attributes.  On 
one  hand,  existing  system  baselines  minimize  development  costs.  On  the  other  hand,  open 
architectures  facilitate  life  cycle  development.  The  current  fiscal  environment  within  DoD  does 
not  favor  system  development  programs  with  a  high  cost  profile.  Nonrecurring  engineering 
funds  are  shrinking.  An  incremental  transition  from  an  existing  system  to  open  architecture 
would  spread  out  die  cost  and  mitigate  the  risk.  The  incremental  approach  also  advances  the  long 
term  goals  of  open  architectures. 

'nie  product  development  methodology  proposed  in  this  paper  helps  attain  this  goal.  Using 
an  existing  system  as  a  baseline,  the  transition  takes  a  low  risk  path  to  incrementally  integrate 
open  architecture  concepts  into  the  complex  computer  system.  Figure  6  depicts  this  concept  for  a 
generic  sensor  processing  chain. 
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Figure  6.  Incremental  Approach  for  Open  Architectures 

The  network  interface  builds  an  open  architecture  upon  the  strong  foundation  of  the  baseline 
sensor  processing  chain.  First,  tap  into  the  processing  chain.  Next,  insert  the  subsets  of  the 
open  architecture  through  the  network  interface  units.  For  example,  evaluate  a  novel  beamformer 
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by  bypassing  the  existing  beamformer.  Gradually  change  the  system  as  technology  becomes 
available  at  a  suitable  cost  Eventually,  the  open  architecture  replaces  the  baseline  sensor  chain. 

The  PDM  facilitates  consideration  of  various  implementations  for  each  functional  block. 
Commercial  off-the-shelf  equipment  serves  as  an  excellent  implementation  for  initial  product 
development.  COTS  equipment  maintains  the  cost  for  initial  test  and  evaluation.  Test  the  COTS 
prototypes  for  shock,  vibration,  temperature,  etc.,  and  deploy  the  equipment  with  acceptable 
performance.  Militarize  the  COTS  equipment  if  environmental  test  results  fall  short  of 
expectations.  Risk  mitigation  occurs  because  the  government  expends  additional  capital  only  for 
verified  performance.  In  addition,  the  availability  of  the  baseline  system  serves  as  a  contingency 
to  reduce  the  risk  of  product  failure. 

Table  I  clarifies  the  utility  of  the  PDM  for  a  next  generation  sensor  system.  In  this  scenario,  a 
sensor  system  already  in  production  serves  as  the  baseline.  Generic  candidates  for  technology 
insertion  includes  today’s  technology,  near  term  upgrades,  and  projected  future  commercial 
technology. 


Processing  Type 

Today's 

Technology 

Technology 

Near  Term  (<1  Year) 

Future  (>5  Years) 

Conventional  Beamformers 

4,000  MIPS 

4,000  MIPS 

40,000  MIPS 

NA 

HK*rcTalTaT:&^ 

mmmsssmsmm mm 

icrfflaga 

—  1  — 

WHStimam 

Data  Processing 

70  MIPS 

70  MIPS 

300  MIPS 

Data  Processing 

35  68030s 

35  68030s 

130  68040s 

Data  Processing  &  I/O 

130  68030s 

130  68030s 

500  68040s 

*  All  units  represent  effective  capability  on  a  per  cabinet  basis. 


Table  I.  PDM  Application  to  Open  Architecture  Technology  Insertion 

Note  the  equipment  undergoing  advanced  development  functions  as  an  excellent  augmentation 
to  the  baseline  system.  The  technology  insertion  category  would  include  near  term  signal 
processing  and  adaptive  beamforming  technologies.  Future  oriented  technologies,  which  include 
parallel  processors  (e.g.,  iWarp,  Paragon,  Connection  Machine)  repackaged  in  multichip 
modules,  do  not  demonstrate  a  definitive  payoff.  The  PDM  suggests  reconsideration  of  these 
technologies  when  the  commercial  market  brings  down  the  cost.  In  the  interim,  develop  the 
network  interface  units  to  facilitate  the  insertion  of  the  advanced  technologies  as  the  market 
matures. 
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SUMMARY 

This  paper  proposes  an  economical  product  development  methodology  for  complex 
computer  systems.  The  PDM  exploits  the  commercial  market  analogy  to  establish  a  flexible,  low 
risk,  cost  effective  approach  for  technological  progress.  The  strategy  pursues  the  state-of-the  art 
while  addressing  risk  management.  The  key  to  the  methodology  is  an  incremental  development 
process. 

The  PDM  assesses  discrete  candidates  and  simplifies  macroscopic  decisions.  The  process 
provides  die  opportunity  to  pursue  the  state-of-the-art  while  concentrating  on  choices  that 
emphasize  low  risk,  cost  savings,  and  short  payback  periods. 

Several  guidelines  enhance  the  chances  of  attaining  this  objective: 

1 .  increase  performance Aec hn ol ogy  in  an  incremental  fashion, 

2 .  use  subjective  decision  techniques  to  eliminate  poor  candidates  from  the  start, 

3 .  fine  tune  detailed  quantitative  analyses  as  the  product  becomes  well  defined, 

4.  create  a  diversified  development  portfolio  directed  at  a  quantum  leap  in  technology, 

5 .  use  COTS  equipment  for  rapid  prototyping, 

6.  build  products  quickly  to  reduce  development  costs,  and 

7 .  cultivate  products  with  short  payback  periods. 

The  methodology  has  particular  application  to  complex  mission  critical  computer  systems. 

An  open  architecture  transition  illustrates  the  utility  of  die  IDP  product  development 
methodology.  The  PDM  sets  priorities  for  a  sensor  processing  chain  by  subjectively  identifying 
the  time,  cost  and  performance  characteristics  of  candidate  technologies. 

System  engineers  can  use  the  product  development  methodology  described  in  this  paper  by: 

1 .  creating  a  detailed  handbook  to  guide  decisions, 

2 .  devising  an  expert  system  to  expedite  the  selection  for  technology  insertion, 

3 .  applying  software  tools  which  refine  the  hierarchical  decision  process,  and 

4.  using  the  PDM  as  a  basis  for  developing  complex  mission  critical  computer  systems. 

The  PDM  integrates  the  choices  input  at  lower  levels  into  higher  level  systems  engineering 
decisions.  System  engineers  could  clearly  define  the  affect  of  the  paths  from  subsystem  to 
system  level  design.  Each  candidate  at  the  lower  level  contributes  to  overall  cost,  performance, 
schedule  and  risk  assessments.  In  this  manner,  the  PDM  enables  an  efficient  approach  for 
systems  engineering. 
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